Mots Visuels Pour Le Calcul De Pose (visual Words for Pose Computation)

Estimating the pose (position and orientation) of a camera with respect to a xed 3D coor-dinate system from an image of the surrounding environment captured in that pose has manyapplications. In our work we try to establish 2D-to-3D point correspondences between the im-age and the environment in order to apply Perspective-n-Point (PnP) algorithm to compute thepose. Each 2D-to-3D point correspondence associates the 2D image coordinates of an imagepoint to the 3D coordinates of the corresponding point in the environment. In order to establisha 2D-to-3D point correspondence through a 3D point captured in an image, we should know(i)3D coordinate of the point and (ii)quanti ed visual characteristics of the point which can beused to identify its 2D location in a given image. We propose a framework in which we com-pute the 3D map i.e. 3D coordinates and visual characteristics of some of the points in theenvironment, through an o ine training stage using a set of training images of the environment.Given a new test image we establish the 2D-to-3D point correspondences by using the 3D mapto detect some of the 3D points visible in the image. During the training stage we perform SfM(Structure from Motion) on training images to compute 3D coordinates of some of the pointsin the environment through . In order to perform SfM we need 2D-tracks of 3D points in thetraining images. Each 2D-track consists of a set of 2D image coordinates of a single 3D point indi erent training images. We establish 2D-track by clustering the SIFT descriptors computedfrom training images. The 2D positions of the SIFT descriptors in a cluster is used to establisha 2D-track. SfM computes the 3D coordinates of the points corresponding to these 2D-tracksby minimizing the reprojection error. During the test stage, the SIFT descriptors associated the2D-track of a 3D point is used to recognize the 3D point in a given image. The overall processis similar to visual word framework used in di erent elds of computer vision. During training,visual word formation is performed through clustering and during testing 3D points are identi edthrough visual word recognition. We experiment with di erent clustering schemes (k-means andmean-shift) and propose a novel scheme for visual word formation. We evaluate di erent aspectsof the quality of 2D-tracks and the 3D points computed from these schemes during training.During test stage, we experiment with di erent matching rules including some of the popularsupervised pattern classi cation methods to perform visual word recognition. We evaluate thevarious recognition strategies based on the accuracy of pose computation under varying degreeof pose di erence between training and test images. In order to achieve robustness against posevariation between train and test images, we explore di erent ways of incorporating SIFT descrip-tors extracted from synthetic views generated from the training images. While experimentingwith mean-shift clustering we conceived a strategy to accelerate its computation by dividing theset of training vectors into groups such that vectors in one group will never in uence the com-putation of clusters in another group. We present the acceleration strategy and mathematicalproof with experimental evaluation.